kourouklidesfandomcom-20200213-history
Dimensionality Reduction
This page contains resources about Dimensionality Reduction, Model Order Reduction , Blind Signal Separation, Source Separation, Subspace Learning, and Continuous Latent Variable Models. Subfields and Concepts * Supervised Dimensionality Reduction ** Linear Discriminant Analysis (LDA) *** Fisher Linear Discriminant (FDA) ** Quadratic Discriminant Analysis (QDA) ** Mixture Discriminant Analysis (MDA) ** Neural Network Matrix Factorization (NNMF) ** Feature Selection *** Bayesian Feature Selection * Unsupervised Dimensionality Reduction ** Singular Value Decomposition (SVD) ** Principal Component Analysis (PCA) / Proper Orthogonal Decomposition (POD) ** Probabilistic PCA (PPCA) ** Canonical-Correlation Analysis ** Independent Component Analysis (ICA) ** Exploratory Factor Analysis (EFA) ** Singular Spectrum Analysis (SSA) ** Empirical Orthogonal Function (EOF) Analysis ** Non-negative Matrix Factorization (NNMF) ** Multinomial PCA ** Truncated SVD / Latent Semantic Analysis / Latent Semantic Indexing ** Maximum-Margin (Minimum-Norm) Matrix Factorization ** Artificial Neural Networks *** Autoencoder **** Linear Autoencoder (equivalent to PCA) **** Stacked Denoising Autoencoder **** Generalized Denoising Autoencoder **** Sparse Autoencoder **** Contractive Autoencoder (CAE) **** Variational Autoencoder (VAE) *** Kohonen Network / Self-organizing map (SOM) / Self-organising feature map (SOFM) ** Unsupervised Deep Learning *** Deep Autoencoder ** K-SVD (used in Dictionary Learning) * Nonlinear Dimensionality Reduction ** Manifold Learning (unsupervised, but supervised variants exist) *** Autoencoder *** SOM / SOFM *** Gaussian Process Latent Variable Model (GPLVM) *** Diffeomorphic Dimensionality Reduction / Diffeomap *** Isomap *** Locally Linear Embedding (LLE) *** Hessian Eigenmapping or Hessian LLE (HLLE) *** Modified Locally-Linear Embedding (MLLE) *** Supervised LLE (SLLE) *** Topologically Constrained Isometric Embedding (TCIE) *** Laplacian Eigenmaps / Spectral Embedding *** Stochastic Proximity Embedding (SPE) *** Local Tangent Space Alignment (LTSA) *** t-distributed stochastic neighbor embedding (t-SNE) *** Local Multidimensional Scaling (MDS) *** Kernel PCA (KPCA) *** Nonlinear PCA (NPCA) *** Nonlinear ICA *** Curvilinear Component Analysis *** Curvilinear Distance Analysis *** Manifold Alignment *** Diffusion Maps *** Maximum Variance Unfolding * Latent Variable Models ** Mixture of Dimensionality Reducers * Canonical Angles / Principal Angles (between subspaces) * Subspace Tracking ** Grassmannian Rank-One Update Subspace Estimation (GROUSE) ** Parallel Estimation and Tracking by REcursive Least Squares (PETRELS) ** Multiscale Online Union of Subspaces Estimation (MOUSSE) ** Grassmannian Robust Adaptive Subspace Tracking Algorithm (GRASTA) ** Online Supervised Dimensionality Reduction (OSDR) Online Courses Video Lectures *Dimensionality Reduction by Neil D. Lawrence *Lecture: Dimensionality reduction Using PCA by S. Sengupta *Lecture: Dimensionality Reduction by David Hogg *Dimensionality Reduction by Feature Selection in Machine Learning by Dunja Mladenić *Subspace Learning by Alessandro Rudi *Lecture: Nonlinear Dimensionality Reduction by Neil D. Lawrence Lecture Notes *Multivariate Analysis, Dimensionality Reduction, and Spectral Methods by Sham Kakade *Large Scale Learning by Sham Kakade and Greg Shakhnarovich *Mathematics for Data Science by Bowei Yan *Dimensionality Reduction by Andrzej Pronobis - with code *Lecture: Dim Reduction by Paris Smaragdis and Sarah E. King *Lecture: Dimension Reduction by Alan L. Yuille *Lecture: Dimensionality Reduction by Oxley Hall *Lecture: Dimensionality reduction (PCA, LDA) by Ricardo Gutierrez-Osuna *Lecture: Dimensionality reduction, Feature selection by Milos Hauskrecht *Lecture: Nonlinear Dimensionality reduction by Milos Hauskrecht *Lecture: Reducing Data Dimension by Tom M. Mitchell *Lecture: Dimensionality Reduction by Andrew Ng *Lecture: Dimensionality reduction by Nuno Vasconcelos *Lecture: Linear dimensionality reduction by Percy Liang *Lecture: Dimensionality Reduction by Sethu Vijayakumar *Lecture: Dimensionality Reduction by Shai Shalev-Shwartz *Lecture: The Curse of Dimensionality and PCA by Olga Veksler *Lecture: Dimensionality Reduction by Gwenn Englebienne *Lecture: Dimensionality reduction by Doina Precup *Lecture: Dimensionality Reduction by Javier Hernandez Rivera *Lecture: Unsupervised Learning by Andrew Zisserman *Lecture: Dimensionality Reduction by Euripides G.M Petrakis *Advanced Statistical Machine Learning by Stefanos Zafeiriou *Model Reduction by David Amsallem & Charbel Farhat Books and Book Chapters * Bengio, Y., Goodfellow, I. J., & Courville, A. (2016). "Chapter 13: Linear Factor Models". Deep Learning. MIT Press. * Theodoridis, S. (2015). "Chapter 19: Dimensionality Reduction". Machine Learning: A Bayesian and Optimization Perspective. Academic Press. * Hastie, T., Tibshirani, R., & Wainwright, M. (2015). "Chapter 7: Matrix Decompositions, Approximations, and Completion". Statistical learning with sparsity: the lasso and generalizations. CRC Press. * Shalev-Shwartz, S., & Ben-David, S. (2014). "Chapter 26: Dimensionality Reduction". Understanding Machine Learning: From Theory to Algorithms. Cambridge University Press. * Sun, L., Ji, S., & Ye, J. (2013). Multi-Label Dimensionality Reduction. CRC Press. * Lu, H., Plataniotis, K. N., & Venetsanopoulos, A. (2013). Multilinear subspace learning: Dimensionality reduction of multidimensional data. CRC press. * Rajaraman, A., & Ullman, J. D. (2012). "Chapter 11: Dimensionality Reduction". Mining of Massive Datasets. Cambridge University Press. * Murphy, K. P. (2012). "Chapter 12: Latent linear models". Machine Learning: A Probabilistic Perspective. MIT Press. * Barber, D. (2012). "Chapter 15: Unsupervised Linear Dimension Reduction". Bayesian Reasoning and Machine Learning. Cambridge University Press. * Barber, D. (2012). "Chapter 16: Supervised Linear Dimension Reduction". Bayesian Reasoning and Machine Learning. Cambridge University Press. * Barber, D. (2012). "Chapter 21: Latent Linear Models". Bayesian Reasoning and Machine Learning. Cambridge University Press. * Alpaydin, E. (2010). "Chapter 6: Dimensionality Reduction". Introduction to machine learning. MIT Press. * Comon, P., & Jutten, C. (Eds.). (2010). Handbook of Blind Source Separation: Independent component analysis and applications. Academic press. * Gorban, A. N., Kégl, B., Wunsch, D. C., & Zinovyev, A. (2008). Principal Manifolds for Data Visualization and Dimension Reduction. Springer. * Ranjan, A. (2008). A'' ''New Approach for Blind Source Separation of Convolutive Sources. VDM Verlag. * Lee, J. A., & Verleysen, M. (2007). Nonlinear Dimensionality Reduction. Springer. * Skillicorn, D. (2007). Understanding complex datasets: data mining with matrix decompositions. CRC press. * Bishop, C. M. (2006). "Chapter 12: Continuous Latent Variables". Pattern Recognition and Machine Learning. Springer. * MacKay, D. J. (2003). "Chapter 34: Independent Component Analysis and Latent Variable Modelling " Information Theory, Inference and Learning Algorithms. Cambridge University Press. Scholarly Articles * Sorzano, C. O. S., Vargas, J., & Montano, A. P. (2014). A survey of dimensionality reduction techniques. arXiv preprint arXiv:1403.2877. * Baur, U., Benner, P., & Feng, L. (2014). Model order reduction for linear and nonlinear systems: a system-theoretic perspective. Archives of Computational Methods in Engineering, 21(4), 331-358. * Gu, C. (2011). Model order reduction of nonlinear dynamical systems. ''PhD Diss., University of California, Berkeley. * Burges, C. J. (2010). ''Dimension Reduction: A Guided Tour. Foundations and Trends® in Machine Learning, 4(3). Now Publishers Inc. * Cunningham, P. (2008). Dimension Reduction. In Machine Learning Techniques for Multimedia (pp. 91-112). Springer. * Fodor, I. K. (2002). A survey of Dimension Reduction Techniques. Tutorials *Dimensionality Reduction by Ali Ghodsi (2006) *Dimensionality Reduction the Probabilistic Way by Neil D. Lawrence (ICML 2008) *Dimensionality Reduction by Wei-Lun Chao (2011) *Dimensionality Reduction From Several Angles by (2013) Software *Dimensionality Reduction (Statistics and Machine Learning Toolbox) - MATLAB *Discriminant Analysis (Statistics and Machine Learning Toolbox) - MATLAB *Toolbox for Dimensionality Reduction (TU Delft) - MATLAB *MATLAB Toolbox for Dimensionality Reduction by Laurens van der Maaten *MATLAB codes for Dimensionality Reduction (Subspace Learning) by Deng Cai *gensim - Python *Dimension Reduction with PCA (scikit-learn) - Python *Multifactor Dimensionality Reduction (MDR) See also * Discrete Latent Variable Models / Clustering, an other set of methods for Unsupervised Learning * Statistical Signal Processing / Adaptive Filter Theory * Optimization * Probabilistic Graphical Models * Dictionary Learning * Deep Learning and Representation Learning * MOR wiki - Model Order Reduction wiki Other Resources *Dimensionality Reduction @ Toronto *Dimensionality reduction for sparse binary data - using gensim Python library Category:Machine Learning Category:Control Theory Category:Signal Processing